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Abstract. Voting is a simple mechanism to combine to- 
gether the preferences of multiple agents. Agents may try to 
manipulate the result of voting by mis-reporting their pref- 
erences. One barrier that might exist to such manipulation 
is computational complexity. In particular, it has been shown 
that it is NP-hard to compute how to manipulate a number of 
different voting rules. However, NP-hardness only bounds the 
worst-case complexity. Recent theoretical results suggest that 
manipulation may often be easy in practice. In this paper, 
we study empirically the manipulability of single transferable 
voting (STV) to determine if computational complexity is re- 
ally a barrier to manipulation. STV was one of the first voting 
rules shown to be NP-hard. It also appears one of the harder 
voting rules to manipulate. We sample a number of distribu- 
tions of votes including uniform and real world elections. In 
almost every election in our experiments, it was easy to com- 
pute how a single agent could manipulate the election or to 
prove that manipulation by a single agent was impossible. 

1 INTRODUCTION 

Agents may try to manipulate an election by mis-reporting 
their preferences in order to get a better result for themselves. 
The Gibbard Satterthwaite theorem proves that, under some 
simple assumptions, there will always exist situations where 
such manipulation is possible |21l I25| . In an influential paper 
[3], Bartholdi, Tovey and Trick proposed an appealing escape: 
perhaps it is computationally so difficult to find a successful 
manipulation that agents have little option but to report their 
true preferences? To illustrate this idea, they demonstrated 
that the second order Copeland rule is NP-hard to manipu- 
late. Shortly after, Bartholdi and Orlin proved that the more 
well known Single Transferable Voting (STV) rule is NP-hard 
to manipulate 2 . A whole subfield of social choice has since 
grown from this proposal, proving that various voting rules 
are NP-hard to manipulate under different assumptions. 

Our focus here is on the manipulability of the STV rule. 
Bartholdi and Orlin argued that STV is one of the most 
promising voting rules to consider in this respect: 

"STV is apparently unique among voting schemes in ac- 
tual use today in that it is computationally resistant to 
manipulation." (page 341 of [2]). 

Whilst there exist other voting rules which are NP-hard to 
manipulate, computational complexity is either restricted to 
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somewhat obscure voting rules like second order Copeland or 
to more well known voting rules but with the rather artificial 
restriction that there are large weights on the votes. STV is 
the only commonly used voting rule that is NP-hard to ma- 
nipulate without weights. STV also appears more difficult to 
manipulate than many other rules. For example. Chamberlain 
studied |5, four different measures of the manipulability of a 
voting rule: the probability that manipulation is possible, the 
number of candidates who can be made to win, the coali- 
tion size necessary to manipulate, and the margin-of-error 
which still results in a successful manipulation. Compared to 
other commonly used rules like plurality and Borda, his re- 
sults showed that STV was the most difficult to manipulate 
by a substantial margin. He concluded that: 

"[this] superior performance . . . combined with the rather 
complex and implausible nature of the strategies to ma- 
nipulate it, suggest that it [the STV rule] may be quite 
resitant to manipulation" (page 203 of [5]). 

Unfortunately, the NP-hardness of manipulating STV is 
only a worst-case result and may not refiect the difficulty of 
manipulation in practice. Indeed, a number of recent theoret- 
ical results suggest that manipulation can often be computa- 
tionally easy on average [HI 1241 1311 1121 132] . Such theoretical 
results typically provide approximation methods so do not say 
what happens with the complete methods studied here (where 
worst case behaviour is exponential). Most recently, Walsh 
has suggested that empirical studies might provide insights 
into the computational complexity of manipulation that can 
complement such theoretical results [30]. However, Walsh's 
empirical study was limited to the simple veto rule, weighted 
votes and elections with only three candidates. In this paper, 
we relax these assumptions and consider the more complex 
multi-round STV rule, unweighted votes, and large numbers 
of candidates. 



2 MANIPULATING STV 

Single Transferable Voting (STV) proceeds in a number of 
rounds. We consider the case of electing a single winner. Each 
agent totally ranks the candidates. Unless one candidate has a 
majority of first place votes, we eliminate the candidate with 
the least number of first place votes. Any ballots placing the 
eliminated candidate in first place are re-assigned to the sec- 
ond place candidate. We then repeat until one candidate has a 
majority. STV is used in a wide variety of elections including 



for the Irish presidency, the Australian House of Representa- 
tives, the Academy awards, and many organizations including 
the American Political Science Association, the International 
Olympic Committee, and the British Labour Party. 

STV is NP-hard to manipulate by a single agent if the num- 
ber of candidates is unbounded and votes are unweighted [2], 
or by a coalition of agents if there are 3 or more candidates and 
votes are weighted [5] . Coleman and Teague give an enumera- 
tive method for a coalition of k unweighted agents to compute 
a manipulation of the STV rule which runs in 0{m\{n + mk)) 
time where n is the number of agents voting and m is the 
number of candidates 0. For a single manipulator, Conitzer, 
Sandholm and Lang give an 0(nl.62"') time algorithm (called 
CSL from now on) to compute the set of candidates that can 
win a STV election [9]. 

In Figure IT] we give a new algorithm for computing a ma- 
nipulation of the STV rule which improves upon CSL in sev- 
eral directions First, our algorithm ignores elections in which 
the chosen candidate is eliminated. Second, our algorithm ter- 
minates search as soon as a manipulation is found in which 
the chosen candidate wins. Third, our algorithm does not ex- 
plore the left branch of the search tree when the right branch 
gives a successful manipulation. 
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Table 1. Comparison between the CSL algorithm and our im- 
proved algorithm to compute a manipulation of a STV election. 

To show the benefits of these improvements, we ran an ex- 
periment in which n agents vote uniformly at random over n 
possible candidates. The experiment was run in CLISP 2.42 
on a 3.2 GHz Pentium 4 with 3GB of memory running Ubuntu 
8.04.3. Table [T] gives the mean nodes explored and runtime 
needed to compute a manipulation or prove none exists. Me- 
dian and other percentiles display similar behaviour. We see 
that our new method can be more than an order of magni- 
tude faster than CSL. In addition, as problems get larger, 
the improvement increases. At n = 32, our method is nearly 
10 times faster than CSL. This increases to roughly 40 times 
faster at n = 128. These improvements reduce the time to find 
a manipulation on the largest problems from several hours to 
a couple of minutes. 



3 UNIFORM VOTES 



We start with one of the simplest possible scenarios: elections 
in which each vote is equally likely. We have one agent try- 
ing to manipulate an election of m candidates where n other 
agents vote. Votes are drawn uniformly at random from all 
m! possible votes. This is the Impartial Culture (IC) model. 



3.1 VARYING THE AGENTS 

In Figures [2] and [3] we plot the probability that a manipulator 
can make a random agent win, and the cost to compute if this 
is possible when we fix the number of candidates but vary 
the number of agents in the election. In this and subsequent 
experiments, we tested 1000 problems at each point. Unless 
otherwise indicated, the number of candidates and of agents 
are varied in powers of 2 from 1 to 128. 
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total number of agents voting, n+1 
Figure 2. Manipulability of random uniform votes. The number 
of candidates is fixed and we vary the number of agents. 

The ability of an agent to manipulate the election decreases 
as the number of agents increases. Only if there are few votes 
and few candidates is there a significant chance that the ma- 
nipulator will be able to change the result. Unlike domains 
like satisfiability [221116) . constraint satisfaction |15U14| . num- 
ber partitioning |18l 120] and the traveling salesperson prob- 
lem |19| . the probability curve does not appear to sharpen to 
a step function around a fixed point. The probability curve 
resembles the smooth phase transitions seen in polynomial 
problems like 2-coloring [l] and l-in-2 satisfiability [29]. Note 
that as elsewhere, we assume that ties are broken in favour 
of the manipulator. For this reason, the probability that an 
election is manipulable is greater than —. 
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Figure 3. Search to compute if an agent can manipulate an elec- 
tion with random uniform votes. The number of candidates is fixed 
and we vary the number of agents. 

Finding a manipulation or proving none is possible is easy 
unless we have both a a large number of agents and a large 
number of candidates. However, in this situation, the chance 
that the manipulator can change the result is very small. 



; Is there one candidate left? 
; Is it the chosen candidate? 
; Is the top of the manipulator's vote currently free? 

; Who will currently he eliminated? 

; Suppose the manipulator votes for them 



; Does this not change the result? 



Manipulate(c, R,{si,..., Sm),f) 

1 if \R\ = 1 

2 then return (R — {c}) 

3 if / = 

4 then 

5 rf <— arg minj^R{sj) 

6 Sd ■^ Sd + w 

7 e -^ arg minj^R{sj) 

8 if d = e 

9 then return 

10 (c / rf) and Manipvlate{c, R — {d}, Trans fer{{si,. 

11 else return 

12 {(c^d) and Manipvlatf.(c, R — {d},Transfer{{si,. .. ,Sm),d,R),0)) or 

13 ((c / e) and Manipvlate(c, R — {e},Transfer{{si,. .. ,Sm),e,R),d)) 

14 else ,• The top of the manipulator's vote is fixed 

15 d <— arg minjizR{sj) ; Who will be eliminated? 

16 if c = d ; Is this the chosen candidate? 

17 then return false 

18 if d = / ; Is the mantpulator free again to change the result? 

19 then return Manipulate(c, i? — {d},rrans/er((si, . . . , s™), d, _R), 0) 

20 else return Manipulate(c, i? — {d},rrans/er((si, ..., Sm), d, _R), /) 



),d,i?),0) 



We use integers from Lto m for the can4idates, integers from 1 to n for the agents (with n being the maniDulatory, ^ ^w 
Figure 1. Our improved algorithni to compute if an agent can manipulate a STV election. 



for the candidate who the 

_ _^ _. j^___ ^ ^_ ^ ^_ __ _ _ . jn. 

manipulator wants to win, R for the set of un-climinatcd candidates, Sj for the weight of agents ranking candidate j first amongst R, w for the 

weight of the manipulator, and / for the candidate most highly ranked by the manipulator amongst R. (or if there is currently no constraint on 

who is most highly ranked). The function Transfer computes the a vector of the new weights of agents ranking candidate j first amongst R after 

a given candidate is eliminated. The algorithm is initially called with R set to every candidate, and / to 0. 



3.2 VARYING THE CANDIDATES 

In FiguresHj we plot the search to compute if the manipulator 
can make a random agent win when we fix the number of 
agents but vary the number of candidates. The probability 
curve that the manipulator can make a random agent win 
resembles Figure [2] 
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Figure 4. Search to compute if an agent can manipulate an elec- 
tion with random uniform voting. The number of agents is fixed 
and we vary the number of candidates. 

Whilst the cost of computing a manipulation increases ex- 
ponential with the number of candidates m, the observed scal- 
ing is much better than the 1.62™. We can easily compute ma- 
nipulations for up to 128 candidates. Note that 1.62™ is over 
lO'^® for m = 128. Thus, we appear to be far from the worst 
case. We fitted the observed data to the model afe™ and found 
a good fit with h = 1.008 and a coefficient of determination, 
R^ = 0.95. 



4 URN MODEL 

In many real life situations, votes are not completely uniform 
and uncorrelated with each other. What happens if we in- 
troduce correlation between votes? Here we consider random 
votes drawn from the Polya Eggenberger urn model [1]. We 
also observed very similar results when votes are drawn at 
random which are single peaked or single troughed. In the 
urn model, we have an urn containing all m! possible votes. 
We draw votes out of the urn at random, and put them back 
into the urn with a additional votes of the same type (where 
a is a parameter). As a increases, there is increasing correla- 
tion between the votes. This generalizes both the Impartial 
Culture model [a — 0) and the Impartial Anonymous Culture 
(a = 1) model. To give a parameter independent of problem 
size, we consider b = -^. For instance, with fe = 1, there is a 
50% chance that the second vote is the same as the first. 

In Figures [5] and [6J we plot the probability that a manipula- 
tor can make a random agent win, and the cost to compute if 
this is possible as we vary the number of candidates in an elec- 
tion where votes are drawn from the Polya Eggenberger urn 
model. The search cost to compute a manipulation increases 
exponential with the number of candidates m. However, we 
can easily compute manipulations for up to 128 candidates 
and agents. We fitted the observed data to the model afc™ 
and found a good fit with 6 = 1.001 and a coefficient of de- 
termination, R^ = 0.99. 

In Figure [7] we plot the cost to compute a manipulation 
when we fix the number of candidates but vary the number 
of agents. As in previous experiments, finding a manipulation 
or proving none exists is easy even if we have many agents 
and candidates. We also saw very similar results when we 
generated single peaked votes using an urn model. 




10 
candidates, m 



Figure 5. Manipulability of correlated votes. The number of 
agents is fixed and we vary thie number of candidates. Tlie n fixed 
votes are drawn from tiie Polya Eggenberger urn model with b = 1. 
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Figure 6. Search to compute if an agent can manipulate an elec- 
tion with correlated votes. The number of agents is fixed and we 
vary the number of candidates. The n fixed votes are drawn us- 
ing the Polya Eggenberger urn model with 6 = 1. The curves for 
different n fit closely on top of each other. 

5 COALITION MANIPULATION 

Our algorithm for computing manipulation by a single agent 
can also be used to compute if a coalition can manipulate an 
election when the members of coalition vote in unison. This 
ignores more complex manipulations where the members of 
the coalition need to vote in different ways. Insisting that the 
members of the coalition vote in unison might be reasonable if 
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Figure 7. Search to compute if an agent can manipulate an elec- 
tion with correlated votes. The number of candidates is fixed and 
we vary the number of agents. The n fixed votes are drawn using 
the Polya Eggenberger urn model with b = 1. 



we wish manipulation to have both a low computational and 
communication cost. In Figures [8] and [9l we plot the probabil- 
ity that a coalition voting in unison can make a random agent 
win, and the cost to compute if this is possible as we vary the 
size of the coalition. Theoretical results in [51] and elsewhere 
suggest that the critical size of a coalition that can just ma- 
nipulate an election grows as y'n. We therefore noriualize the 
coalition size by ^/n. 
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normalized coalition size, k/sqrt(n) 

Figure 8. Manipulability of an election as we vary the size of the 

manipulating coalition. The number of candidates is the same as 

the number of non-manipulating agents. 

The ability of the coalition to manipulate the election in- 
creases as the size of the coalition increases. When the coali- 
tion is about -^/n in size, the probability that the coalition can 
manipulate the election so that a candidate chosen at random 
wins is around |. The cost to compute a manipulation (or 
prove that none exists) decreases as we increase the size of 
the coalition. It is easier for a larger coalition to manipulate 
an election than a smaller one. 
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Figure 9. Search to compute if a coalition can manipulate an 

election as we vary coalition size. 

These experiments again suggest different behaviour occurs 
here than in other combinatorial problems like propositional 
satisfiability and graph colouring [6l 1261 l27l 128) . For instance, 
we do not see a rapid transition that sharpens around a fixed 
point as in 3-satisfiability [^. When we vary the coalition 
size, we see a transition in the probability of being able to 
manipulate the result around a coalition size k — ^/n. How- 
ever, this transition appears smooth and does not seem to 
sharpen towards a step function as n increases. In addition, 
hard instances do not occur around k = ^fn. Indeed, the hard- 
est instances are when the coalition is smaller than this and 



has only a small chance of being able to manipulate the result. 

6 SAMPLING REAL ELECTIONS 

Elections met in practice may differ from those sampled so 
far. There might, for instance, be some votes which are never 
cast. On the other hand, with the models studied so far every 
possible random/single peaked vote has a non-zero probability 
of being seen. We therefore sampled some real voting records 

EH 231. 
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Figure 10. Search to compute if an agent can manipulate an elec- 
tion with votes sampled from the NASA experiment. The number 
of agents is fixed and we vary the number of candidates. 
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Figure 11. Search to compute if an agent can manipulate an elec- 
tion with votes sampled from the NASA experiment. The number 
of candidates is fixed and we vary the number of agents. 

Our first experiment uses the votes cast by 10 teams of sci- 
entists to select one of 32 different trajectories for NASA's 
Mariner spacecraft [TT]. Each team ranked the different tra- 
jectories based on their scientific value. We sampled these 
votes. For elections with f or fewer agents voting, we simply 
took a random subset of the 10 votes. For elections with more 
than 10 agents voting, we simply sampled from the 10 votes 
with uniform frequency. For elections with 32 or fewer candi- 
dates, we simply took a random subset of the 32 candidates. 
Finally for elections with more than 32 candidates, we dupli- 
cated each candidate and assigned them the same ranking. 
Since STV works on total orders, we then forced each agent 
to break any ties randomly. 

In Figures [Tol to 1 11 [ we plot the cost to compute if a manip- 
ulator can make a random agent win as we vary the number 
of candidates and agents. Votes are sampled from the NASA 



experiment as explained earlier. The probability that the ma- 
nipulator can manipulate the election resembles the proba- 
bility curve for uniform random votes. The search needed to 
compute a manipulation again increases exponential with the 
number of candidates m. However, the observed scaling is 
much better than 1.62™. We can easily compute manipula- 
tions for up to 128 candidates and agents. 

In our second experiment, we used votes from a faculty 
hiring committee at the University of California at Irvine [T0| . 
We sampled from this data set in the same ways as from the 
NASA dataset and observed very similar results. It was easy 
to find a manipulation or prove that none exists. The observed 
scaling was again much better than 1.62'". 

7 RELATED WORK 

As indicated, there have been several theoretical results re- 
cently that suggest elections are easy to manipulate in prac- 
tice despite NP-hardness results. For example, Procaccia and 
Rosenschein proved that for most scoring rules and a wide 
variety of distributions over votes, when the size of the coali- 
tion is o{^/n), the probability that they can change the result 
tends to 0, and when it is u}{y/n), the probability that they 
can manipulate the result tends to 1 [22 . They also gave a 
simple greedy procedure that will find a manipulation of a 
scoring rule in polynomial time with a probability of failure 
that is an inverse polynomial in n [24J. 

As a second example, Xia and Conitzer have shown that 
for a large class of voting rules including STV, as the number 
of agents grows, either the probability that a coalition can 
manipulate the result is very small (as the coalition is too 
small), or the probability that they can easily manipulate the 
result to make any alternative win is very large [3T]. They 
left open only a small interval in the size for the coalition 
for which the coalition is large enough to manipulate but not 
obviously large enough to manipulate the result easily. 

Friedgut, Kalai and Nisan proved that if the voting rule is 
neutral and far from dictatorial and there are 3 candidates 
then there exists an agent for whom a random manipulation 
succeeds with probability il(-) [12]. Starting from different 
assumptions, Xia and Conitzer showed that a random manip- 
ulation would succeed with probability f2(-) for 3 or more 
candidates for STV, for 4 or more candidates for any scoring 
rule and for 5 or more candidates for Copeland |32| . 

Walsh empirically studied manipulation of the veto rule 
by a coalition of agents whose votes were weighted [30]. He 
showed that there was a smooth transition in the probability 
that a coalition can elect a desired candidate as the size of the 
manipulating coalition increases. He also showed that it was 
easy to find manipulations of the veto rule or prove that none 
exist for many independent and identically distributed votes 
even when the coalition was critical in size. He was able to 
identify a situation in which manipulation was computation- 
ally hard. This is when votes are highly correlated and the 
election is "hung" . However, even a single uncorrelated agent 
was enough to make manipulation easy again. 

Coleman and Teague proposed algorithms to compute a 
manipulation for the STV rule [7]. They also conducted an 
empirical study which demonstrates that only relatively small 
coalitions are needed to change the elimination order of the 
STV rule. They observed that most uniform and random 



elections are not trivially manipulable using a simple greedy 
heuristic. On the other hand, our results suggest that, for ma- 
nipulation by a single agent, a limited amount of backtracking 
is needed to find a manipulation or prove that none exists. 

8 CONCLUSIONS 

We have studied empirically whether computational complex- 
ity is a barrier to the manipulation for the STV rule. We have 
looked at a number of different distributions of votes includ- 
ing uniform random votes, correlated votes drawn from an 
urn model, and votes sampled from some real world elections. 
We have looked at manipulation by both a single agent, and 
a coalition of agents who vote in unison. Almost every one 
of the millions of elections in our experiments was easy to 
manipulate or to prove could not be manipulated. These re- 
sults increase the concerns that computational complexity is 
indeed a barrier to manipulation in practice. 
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